{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1、回归训练区间\n",
    "\n",
    "### 2014-01-01  ~  2016-12-31\n",
    "\n",
    "## 2、回归股票选择\n",
    "\n",
    "### - 股票选择（HS300指数）\n",
    "### - 如果是以A股选择回归，那么回测就以A股去进行选股\n",
    "\n",
    "## 3、回归因子数据准备、收益率计算\n",
    "\n",
    "### - 因子数据：横截面数据拼接，添加日期数据、去除空值\n",
    "### - 收益率计算: 所有样本的收益率计算、取出价格为空值的计算\n",
    "### - 这个回归就是每个月用股票的回报率对股票上一期末的因子值做一个横截 面的回归。 \n",
    "\n",
    "## 4、目标值特征值提取进行回归估计\n",
    "\n",
    "### 数据处理：去除收益为0(价格数据不存在)、去极值、标准化处理、市值中性化处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 一、 准备好这些因子对应日期的数据\n",
    "* 每个月月末的因子数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 确定每月日期 2014-01-01~2016-01-01\n",
    "dates = get_trading_dates(start_date=\"2014-01-01\", end_date=\"2016-01-01\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 每天日期---->每月月末\n",
    "# 每月最后一个交易日, 按月计算收益率\n",
    "month_date = []\n",
    "for i in range(len(dates) -1):\n",
    "  if dates[i].year != dates[i+1].year:\n",
    "    month_date.append(dates[i])\n",
    "  elif dates[i].month != dates[i+1].month:\n",
    "    month_date.append(dates[i])\n",
    "\n",
    "#把最后一个交易日加入\n",
    "month_date.append(dates[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[datetime.date(2014, 1, 30),\n",
       " datetime.date(2014, 2, 28),\n",
       " datetime.date(2014, 3, 31),\n",
       " datetime.date(2014, 4, 30),\n",
       " datetime.date(2014, 5, 30),\n",
       " datetime.date(2014, 6, 30),\n",
       " datetime.date(2014, 7, 31),\n",
       " datetime.date(2014, 8, 29),\n",
       " datetime.date(2014, 9, 30),\n",
       " datetime.date(2014, 10, 31),\n",
       " datetime.date(2014, 11, 28),\n",
       " datetime.date(2014, 12, 31),\n",
       " datetime.date(2015, 1, 30),\n",
       " datetime.date(2015, 2, 27),\n",
       " datetime.date(2015, 3, 31),\n",
       " datetime.date(2015, 4, 30),\n",
       " datetime.date(2015, 5, 29),\n",
       " datetime.date(2015, 6, 30),\n",
       " datetime.date(2015, 7, 31),\n",
       " datetime.date(2015, 8, 31),\n",
       " datetime.date(2015, 9, 30),\n",
       " datetime.date(2015, 10, 30),\n",
       " datetime.date(2015, 11, 30),\n",
       " datetime.date(2015, 12, 31)]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 每月日期列表获取\n",
    "month_date"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 二、准备因子数据根据训练每月最后交易日日期列表（特征值）\n",
    "* 特征值都是该月的因子数据（避免下个月在日期列表当中不存在）\n",
    "* 因子的数据处理：直接删除缺失值\n",
    "* month_date当中datetime.date(2015, 12, 31)号因子数据不用去\n",
    "* [datetime.date(2014, 1, 30),\n",
    " datetime.date(2014, 2, 28),\n",
    " datetime.date(2014, 3, 31),\n",
    " datetime.date(2014, 4, 30),\n",
    " datetime.date(2014, 5, 30),\n",
    " datetime.date(2014, 6, 30),\n",
    " datetime.date(2014, 7, 31),\n",
    " datetime.date(2014, 8, 29),\n",
    " datetime.date(2014, 9, 30),\n",
    " datetime.date(2014, 10, 31),\n",
    " datetime.date(2014, 11, 28),\n",
    " datetime.date(2014, 12, 31),\n",
    " datetime.date(2015, 1, 30),\n",
    " datetime.date(2015, 2, 27),\n",
    " datetime.date(2015, 3, 31),\n",
    " datetime.date(2015, 4, 30),\n",
    " datetime.date(2015, 5, 29),\n",
    " datetime.date(2015, 6, 30),\n",
    " datetime.date(2015, 7, 31),\n",
    " datetime.date(2015, 8, 31),\n",
    " datetime.date(2015, 9, 30),\n",
    " datetime.date(2015, 10, 30),\n",
    " datetime.date(2015, 11, 30),"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 获取沪深300 的股票列表\n",
    "stocks = index_components(\"000300.XSHG\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "all_data = pd.DataFrame()\n",
    "for date in month_date[:-1]:\n",
    "  \n",
    "  q = query(fundamentals.eod_derivative_indicator.pe_ratio, \n",
    "            fundamentals.eod_derivative_indicator.pb_ratio, \n",
    "            fundamentals.eod_derivative_indicator.market_cap, \n",
    "            fundamentals.financial_indicator.ev, \n",
    "            fundamentals.financial_indicator.return_on_asset_net_profit, \n",
    "            fundamentals.financial_indicator.du_return_on_equity, \n",
    "            fundamentals.financial_indicator.earnings_per_share, \n",
    "            fundamentals.income_statement.revenue, \n",
    "            fundamentals.income_statement.total_expense).filter(fundamentals.stockcode.in_(stocks))\n",
    "\n",
    "  # 查询因子数据\n",
    "  fund = get_fundamentals(q, entry_date=date)[:, 0, :]\n",
    "  \n",
    "  fund['date'] = date\n",
    "  \n",
    "  # 进行每月因子数据拼接\n",
    "  all_data = pd.concat([all_data, fund])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 删除缺失值\n",
    "all_data = all_data.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 建立每个股票样本，每个月的因子值对应下个月的收益率\n",
    "all_data['next_month_return'] = np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pe_ratio</th>\n",
       "      <th>pb_ratio</th>\n",
       "      <th>market_cap</th>\n",
       "      <th>ev</th>\n",
       "      <th>return_on_asset_net_profit</th>\n",
       "      <th>du_return_on_equity</th>\n",
       "      <th>earnings_per_share</th>\n",
       "      <th>revenue</th>\n",
       "      <th>total_expense</th>\n",
       "      <th>date</th>\n",
       "      <th>next_month_return</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>000001.XSHE</th>\n",
       "      <td>7.126</td>\n",
       "      <td>0.9684</td>\n",
       "      <td>1.08537e+11</td>\n",
       "      <td>1.85809e+12</td>\n",
       "      <td>0.6756</td>\n",
       "      <td>13.0164</td>\n",
       "      <td>1.43</td>\n",
       "      <td>3.7345e+10</td>\n",
       "      <td>2.2038e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000002.XSHE</th>\n",
       "      <td>5.3769</td>\n",
       "      <td>1.0571</td>\n",
       "      <td>8.12905e+10</td>\n",
       "      <td>1.91646e+11</td>\n",
       "      <td>1.7804</td>\n",
       "      <td>9.3272</td>\n",
       "      <td>0.56</td>\n",
       "      <td>6.34153e+10</td>\n",
       "      <td>5.396e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000008.XSHE</th>\n",
       "      <td>114.328</td>\n",
       "      <td>4.6166</td>\n",
       "      <td>2.60187e+09</td>\n",
       "      <td>2.91761e+09</td>\n",
       "      <td>2.7598</td>\n",
       "      <td>3.0992</td>\n",
       "      <td>0.0561</td>\n",
       "      <td>2.28189e+08</td>\n",
       "      <td>2.06793e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000060.XSHE</th>\n",
       "      <td>28.8896</td>\n",
       "      <td>1.9053</td>\n",
       "      <td>1.13668e+10</td>\n",
       "      <td>1.91828e+10</td>\n",
       "      <td>0.9316</td>\n",
       "      <td>3.068</td>\n",
       "      <td>0.08</td>\n",
       "      <td>9.75875e+09</td>\n",
       "      <td>9.64269e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000063.XSHE</th>\n",
       "      <td>8.2098</td>\n",
       "      <td>2.0244</td>\n",
       "      <td>4.56162e+10</td>\n",
       "      <td>9.63637e+10</td>\n",
       "      <td>0.5568</td>\n",
       "      <td>2.561</td>\n",
       "      <td>0.16</td>\n",
       "      <td>5.45575e+10</td>\n",
       "      <td>5.70062e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            pe_ratio pb_ratio   market_cap           ev  \\\n",
       "000001.XSHE    7.126   0.9684  1.08537e+11  1.85809e+12   \n",
       "000002.XSHE   5.3769   1.0571  8.12905e+10  1.91646e+11   \n",
       "000008.XSHE  114.328   4.6166  2.60187e+09  2.91761e+09   \n",
       "000060.XSHE  28.8896   1.9053  1.13668e+10  1.91828e+10   \n",
       "000063.XSHE   8.2098   2.0244  4.56162e+10  9.63637e+10   \n",
       "\n",
       "            return_on_asset_net_profit du_return_on_equity earnings_per_share  \\\n",
       "000001.XSHE                     0.6756             13.0164               1.43   \n",
       "000002.XSHE                     1.7804              9.3272               0.56   \n",
       "000008.XSHE                     2.7598              3.0992             0.0561   \n",
       "000060.XSHE                     0.9316               3.068               0.08   \n",
       "000063.XSHE                     0.5568               2.561               0.16   \n",
       "\n",
       "                 revenue total_expense        date  next_month_return  \n",
       "000001.XSHE   3.7345e+10    2.2038e+10  2014-01-30                NaN  \n",
       "000002.XSHE  6.34153e+10     5.396e+10  2014-01-30                NaN  \n",
       "000008.XSHE  2.28189e+08   2.06793e+08  2014-01-30                NaN  \n",
       "000060.XSHE  9.75875e+09   9.64269e+09  2014-01-30                NaN  \n",
       "000063.XSHE  5.45575e+10   5.70062e+10  2014-01-30                NaN  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 三、获取价格数据计算对应的收益率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 1、获取每月月末价格数据\n",
    "all_price= pd.DataFrame()\n",
    "for date in month_date:\n",
    "  price = get_price(stocks, start_date=date, end_date=date, fields='close')\n",
    "  all_price = pd.concat([all_price, price], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 转置方便后面计算处理\n",
    "all_price = all_price.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 按照股票进行去除空值\n",
    "all_price = all_price.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/ricequant/lib/python3.5/site-packages/pandas/core/indexing.py:561: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self.obj[item_labels[indexer[info_axis]]] = value\n"
     ]
    }
   ],
   "source": [
    "# 2、计算收益率\n",
    "for i in range(len(all_price.columns)-1):\n",
    "  # 利用每次后一个月的收盘价-循环的这个月的收盘价/循环的这个月的收盘价\n",
    "  all_price.iloc[:, i] = all_price.iloc[:, i + 1]/ all_price.iloc[:, i] - 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2014-01-30 00:00:00</th>\n",
       "      <th>2014-02-28 00:00:00</th>\n",
       "      <th>2014-03-31 00:00:00</th>\n",
       "      <th>2014-04-30 00:00:00</th>\n",
       "      <th>2014-05-30 00:00:00</th>\n",
       "      <th>2014-06-30 00:00:00</th>\n",
       "      <th>2014-07-31 00:00:00</th>\n",
       "      <th>2014-08-29 00:00:00</th>\n",
       "      <th>2014-09-30 00:00:00</th>\n",
       "      <th>2014-10-31 00:00:00</th>\n",
       "      <th>...</th>\n",
       "      <th>2015-03-31 00:00:00</th>\n",
       "      <th>2015-04-30 00:00:00</th>\n",
       "      <th>2015-05-29 00:00:00</th>\n",
       "      <th>2015-06-30 00:00:00</th>\n",
       "      <th>2015-07-31 00:00:00</th>\n",
       "      <th>2015-08-31 00:00:00</th>\n",
       "      <th>2015-09-30 00:00:00</th>\n",
       "      <th>2015-10-30 00:00:00</th>\n",
       "      <th>2015-11-30 00:00:00</th>\n",
       "      <th>2015-12-31 00:00:00</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>600018.XSHG</th>\n",
       "      <td>-0.048302</td>\n",
       "      <td>0.004241</td>\n",
       "      <td>-0.010535</td>\n",
       "      <td>-0.044675</td>\n",
       "      <td>-0.008911</td>\n",
       "      <td>0.053749</td>\n",
       "      <td>0.019745</td>\n",
       "      <td>0.139778</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.084899</td>\n",
       "      <td>...</td>\n",
       "      <td>0.300119</td>\n",
       "      <td>-0.062147</td>\n",
       "      <td>-0.146711</td>\n",
       "      <td>-0.092291</td>\n",
       "      <td>0.091926</td>\n",
       "      <td>-0.110972</td>\n",
       "      <td>0.024396</td>\n",
       "      <td>-0.058822</td>\n",
       "      <td>-0.035720</td>\n",
       "      <td>6.1469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>600061.XSHG</th>\n",
       "      <td>-0.032409</td>\n",
       "      <td>0.093801</td>\n",
       "      <td>-0.035219</td>\n",
       "      <td>-0.003182</td>\n",
       "      <td>-0.020693</td>\n",
       "      <td>0.047157</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.358673</td>\n",
       "      <td>...</td>\n",
       "      <td>0.086844</td>\n",
       "      <td>0.016461</td>\n",
       "      <td>0.275977</td>\n",
       "      <td>-0.454785</td>\n",
       "      <td>-0.263339</td>\n",
       "      <td>-0.028961</td>\n",
       "      <td>0.216947</td>\n",
       "      <td>0.379384</td>\n",
       "      <td>0.047256</td>\n",
       "      <td>25.4813</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>600008.XSHG</th>\n",
       "      <td>0.080235</td>\n",
       "      <td>-0.019302</td>\n",
       "      <td>-0.024249</td>\n",
       "      <td>-0.030913</td>\n",
       "      <td>0.014803</td>\n",
       "      <td>0.027496</td>\n",
       "      <td>0.051955</td>\n",
       "      <td>0.152696</td>\n",
       "      <td>-0.011680</td>\n",
       "      <td>0.022330</td>\n",
       "      <td>...</td>\n",
       "      <td>0.045944</td>\n",
       "      <td>-0.054110</td>\n",
       "      <td>-0.024480</td>\n",
       "      <td>-0.236227</td>\n",
       "      <td>-0.127752</td>\n",
       "      <td>-0.078447</td>\n",
       "      <td>0.233831</td>\n",
       "      <td>-0.030353</td>\n",
       "      <td>-0.033211</td>\n",
       "      <td>4.8644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601877.XSHG</th>\n",
       "      <td>0.083473</td>\n",
       "      <td>-0.061325</td>\n",
       "      <td>-0.030090</td>\n",
       "      <td>-0.090630</td>\n",
       "      <td>0.039862</td>\n",
       "      <td>0.135560</td>\n",
       "      <td>-0.023716</td>\n",
       "      <td>0.062491</td>\n",
       "      <td>-0.009088</td>\n",
       "      <td>0.135165</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.056190</td>\n",
       "      <td>0.033766</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.004727</td>\n",
       "      <td>-0.094979</td>\n",
       "      <td>-0.055311</td>\n",
       "      <td>24.8049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>600482.XSHG</th>\n",
       "      <td>0.150253</td>\n",
       "      <td>-0.013242</td>\n",
       "      <td>-0.103758</td>\n",
       "      <td>0.014852</td>\n",
       "      <td>0.059397</td>\n",
       "      <td>0.095328</td>\n",
       "      <td>0.137375</td>\n",
       "      <td>0.187547</td>\n",
       "      <td>-0.068224</td>\n",
       "      <td>-0.006099</td>\n",
       "      <td>...</td>\n",
       "      <td>0.168191</td>\n",
       "      <td>0.846886</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.097903</td>\n",
       "      <td>0.160608</td>\n",
       "      <td>0.060402</td>\n",
       "      <td>46.6912</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             2014-01-30  2014-02-28  2014-03-31  2014-04-30  2014-05-30  \\\n",
       "600018.XSHG   -0.048302    0.004241   -0.010535   -0.044675   -0.008911   \n",
       "600061.XSHG   -0.032409    0.093801   -0.035219   -0.003182   -0.020693   \n",
       "600008.XSHG    0.080235   -0.019302   -0.024249   -0.030913    0.014803   \n",
       "601877.XSHG    0.083473   -0.061325   -0.030090   -0.090630    0.039862   \n",
       "600482.XSHG    0.150253   -0.013242   -0.103758    0.014852    0.059397   \n",
       "\n",
       "             2014-06-30  2014-07-31  2014-08-29  2014-09-30  2014-10-31  \\\n",
       "600018.XSHG    0.053749    0.019745    0.139778    0.000000    0.084899   \n",
       "600061.XSHG    0.047157    0.000000    0.000000    0.000000    1.358673   \n",
       "600008.XSHG    0.027496    0.051955    0.152696   -0.011680    0.022330   \n",
       "601877.XSHG    0.135560   -0.023716    0.062491   -0.009088    0.135165   \n",
       "600482.XSHG    0.095328    0.137375    0.187547   -0.068224   -0.006099   \n",
       "\n",
       "                ...      2015-03-31  2015-04-30  2015-05-29  2015-06-30  \\\n",
       "600018.XSHG     ...        0.300119   -0.062147   -0.146711   -0.092291   \n",
       "600061.XSHG     ...        0.086844    0.016461    0.275977   -0.454785   \n",
       "600008.XSHG     ...        0.045944   -0.054110   -0.024480   -0.236227   \n",
       "601877.XSHG     ...       -0.056190    0.033766    0.000000    0.000000   \n",
       "600482.XSHG     ...        0.168191    0.846886    0.000000    0.000000   \n",
       "\n",
       "             2015-07-31  2015-08-31  2015-09-30  2015-10-30  2015-11-30  \\\n",
       "600018.XSHG    0.091926   -0.110972    0.024396   -0.058822   -0.035720   \n",
       "600061.XSHG   -0.263339   -0.028961    0.216947    0.379384    0.047256   \n",
       "600008.XSHG   -0.127752   -0.078447    0.233831   -0.030353   -0.033211   \n",
       "601877.XSHG    0.000000    0.000000    0.004727   -0.094979   -0.055311   \n",
       "600482.XSHG    0.000000    0.000000   -0.097903    0.160608    0.060402   \n",
       "\n",
       "             2015-12-31  \n",
       "600018.XSHG      6.1469  \n",
       "600061.XSHG     25.4813  \n",
       "600008.XSHG      4.8644  \n",
       "601877.XSHG     24.8049  \n",
       "600482.XSHG     46.6912  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_price.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 四、将收益率填充到因子对应的下个月收益率列当中\n",
    "* all_data, all_price"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for i in range(len(all_data)):\n",
    "  \n",
    "  # 每个样本填充对应收益率\n",
    "  stock = all_data.index[i]\n",
    "  \n",
    "  date = all_data.ix[i, 'date']\n",
    "  \n",
    "  # 在all_price里面寻找收益率\n",
    "  if stock in all_price.index and date in all_price.columns:\n",
    "    \n",
    "    all_data.ix[i, 'next_month_return'] = all_price.loc[stock, date]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 把收益率为空删除\n",
    "all_data = all_data.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pe_ratio</th>\n",
       "      <th>pb_ratio</th>\n",
       "      <th>market_cap</th>\n",
       "      <th>ev</th>\n",
       "      <th>return_on_asset_net_profit</th>\n",
       "      <th>du_return_on_equity</th>\n",
       "      <th>earnings_per_share</th>\n",
       "      <th>revenue</th>\n",
       "      <th>total_expense</th>\n",
       "      <th>date</th>\n",
       "      <th>next_month_return</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>000001.XSHE</th>\n",
       "      <td>7.126</td>\n",
       "      <td>0.9684</td>\n",
       "      <td>1.08537e+11</td>\n",
       "      <td>1.85809e+12</td>\n",
       "      <td>0.6756</td>\n",
       "      <td>13.0164</td>\n",
       "      <td>1.43</td>\n",
       "      <td>3.7345e+10</td>\n",
       "      <td>2.2038e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.023689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000002.XSHE</th>\n",
       "      <td>5.3769</td>\n",
       "      <td>1.0571</td>\n",
       "      <td>8.12905e+10</td>\n",
       "      <td>1.91646e+11</td>\n",
       "      <td>1.7804</td>\n",
       "      <td>9.3272</td>\n",
       "      <td>0.56</td>\n",
       "      <td>6.34153e+10</td>\n",
       "      <td>5.396e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.089430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000008.XSHE</th>\n",
       "      <td>114.328</td>\n",
       "      <td>4.6166</td>\n",
       "      <td>2.60187e+09</td>\n",
       "      <td>2.91761e+09</td>\n",
       "      <td>2.7598</td>\n",
       "      <td>3.0992</td>\n",
       "      <td>0.0561</td>\n",
       "      <td>2.28189e+08</td>\n",
       "      <td>2.06793e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.121334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000060.XSHE</th>\n",
       "      <td>28.8896</td>\n",
       "      <td>1.9053</td>\n",
       "      <td>1.13668e+10</td>\n",
       "      <td>1.91828e+10</td>\n",
       "      <td>0.9316</td>\n",
       "      <td>3.068</td>\n",
       "      <td>0.08</td>\n",
       "      <td>9.75875e+09</td>\n",
       "      <td>9.64269e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.079866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000063.XSHE</th>\n",
       "      <td>8.2098</td>\n",
       "      <td>2.0244</td>\n",
       "      <td>4.56162e+10</td>\n",
       "      <td>9.63637e+10</td>\n",
       "      <td>0.5568</td>\n",
       "      <td>2.561</td>\n",
       "      <td>0.16</td>\n",
       "      <td>5.45575e+10</td>\n",
       "      <td>5.70062e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.009043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000069.XSHE</th>\n",
       "      <td>7.1736</td>\n",
       "      <td>1.4932</td>\n",
       "      <td>3.56303e+10</td>\n",
       "      <td>6.55072e+10</td>\n",
       "      <td>4.0417</td>\n",
       "      <td>13.7533</td>\n",
       "      <td>0.399</td>\n",
       "      <td>1.72365e+10</td>\n",
       "      <td>1.32602e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.077550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000100.XSHE</th>\n",
       "      <td>6.2327</td>\n",
       "      <td>1.5054</td>\n",
       "      <td>2.13287e+10</td>\n",
       "      <td>5.31028e+10</td>\n",
       "      <td>2.1248</td>\n",
       "      <td>10.2132</td>\n",
       "      <td>0.149</td>\n",
       "      <td>6.12241e+10</td>\n",
       "      <td>6.05763e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.112002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000157.XSHE</th>\n",
       "      <td>111.858</td>\n",
       "      <td>0.935</td>\n",
       "      <td>3.89151e+10</td>\n",
       "      <td>7.0563e+10</td>\n",
       "      <td>4.387</td>\n",
       "      <td>9.2131</td>\n",
       "      <td>0.49</td>\n",
       "      <td>2.88971e+10</td>\n",
       "      <td>2.42468e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.009907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000333.XSHE</th>\n",
       "      <td>10.7138</td>\n",
       "      <td>2.4057</td>\n",
       "      <td>7.90211e+10</td>\n",
       "      <td>9.53968e+10</td>\n",
       "      <td>7.2895</td>\n",
       "      <td>17.3695</td>\n",
       "      <td>3.73</td>\n",
       "      <td>9.39206e+10</td>\n",
       "      <td>8.7225e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.169011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000338.XSHE</th>\n",
       "      <td>8.4296</td>\n",
       "      <td>1.262</td>\n",
       "      <td>3.49879e+10</td>\n",
       "      <td>5.65886e+10</td>\n",
       "      <td>4.0573</td>\n",
       "      <td>10.4072</td>\n",
       "      <td>1.35</td>\n",
       "      <td>4.33327e+10</td>\n",
       "      <td>4.00661e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.002849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000402.XSHE</th>\n",
       "      <td>4.2384</td>\n",
       "      <td>0.6684</td>\n",
       "      <td>1.48932e+10</td>\n",
       "      <td>4.74655e+10</td>\n",
       "      <td>2.1432</td>\n",
       "      <td>6.6307</td>\n",
       "      <td>0.44</td>\n",
       "      <td>1.01632e+10</td>\n",
       "      <td>8.23405e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.065039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000413.XSHE</th>\n",
       "      <td>34.8362</td>\n",
       "      <td>3.5709</td>\n",
       "      <td>2.076e+10</td>\n",
       "      <td>1.65766e+10</td>\n",
       "      <td>4.8108</td>\n",
       "      <td>7.0997</td>\n",
       "      <td>0.32</td>\n",
       "      <td>5.3758e+08</td>\n",
       "      <td>3.29781e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.130489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000415.XSHE</th>\n",
       "      <td>7.9511</td>\n",
       "      <td>1.8681</td>\n",
       "      <td>1.05729e+10</td>\n",
       "      <td>3.54686e+10</td>\n",
       "      <td>1.7595</td>\n",
       "      <td>3.9744</td>\n",
       "      <td>0.4874</td>\n",
       "      <td>4.49556e+09</td>\n",
       "      <td>3.62487e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.005992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000423.XSHE</th>\n",
       "      <td>17.3139</td>\n",
       "      <td>4.7263</td>\n",
       "      <td>2.36429e+10</td>\n",
       "      <td>2.6466e+10</td>\n",
       "      <td>15.7904</td>\n",
       "      <td>19.3468</td>\n",
       "      <td>1.3158</td>\n",
       "      <td>2.85154e+09</td>\n",
       "      <td>1.83781e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.012725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000425.XSHE</th>\n",
       "      <td>24.4981</td>\n",
       "      <td>0.7322</td>\n",
       "      <td>1.41918e+10</td>\n",
       "      <td>3.43816e+10</td>\n",
       "      <td>2.3874</td>\n",
       "      <td>5.8692</td>\n",
       "      <td>0.51</td>\n",
       "      <td>1.86193e+10</td>\n",
       "      <td>1.74667e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.010169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000503.XSHE</th>\n",
       "      <td>11233.5</td>\n",
       "      <td>16.6486</td>\n",
       "      <td>2.09156e+10</td>\n",
       "      <td>1.83114e+10</td>\n",
       "      <td>0.0588</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>0.0023</td>\n",
       "      <td>1.51521e+08</td>\n",
       "      <td>1.35794e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.048560</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000538.XSHE</th>\n",
       "      <td>20.8343</td>\n",
       "      <td>7.0612</td>\n",
       "      <td>6.37545e+10</td>\n",
       "      <td>8.18656e+10</td>\n",
       "      <td>15.1017</td>\n",
       "      <td>22.5812</td>\n",
       "      <td>2.53</td>\n",
       "      <td>1.12287e+10</td>\n",
       "      <td>9.89682e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.064576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000540.XSHE</th>\n",
       "      <td>3.9358</td>\n",
       "      <td>1.9948</td>\n",
       "      <td>6.78551e+09</td>\n",
       "      <td>2.09298e+10</td>\n",
       "      <td>2.5329</td>\n",
       "      <td>26.9999</td>\n",
       "      <td>0.5658</td>\n",
       "      <td>5.37724e+09</td>\n",
       "      <td>4.45218e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.064532</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000559.XSHE</th>\n",
       "      <td>16.8707</td>\n",
       "      <td>2.9299</td>\n",
       "      <td>1.01172e+10</td>\n",
       "      <td>1.26371e+10</td>\n",
       "      <td>3.5344</td>\n",
       "      <td>7.6187</td>\n",
       "      <td>0.184</td>\n",
       "      <td>6.80359e+09</td>\n",
       "      <td>6.42877e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.885040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000568.XSHE</th>\n",
       "      <td>9.2359</td>\n",
       "      <td>2.1721</td>\n",
       "      <td>2.29549e+10</td>\n",
       "      <td>2.99001e+10</td>\n",
       "      <td>19.3343</td>\n",
       "      <td>27.4698</td>\n",
       "      <td>1.93</td>\n",
       "      <td>8.15632e+09</td>\n",
       "      <td>4.54271e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.026879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000623.XSHE</th>\n",
       "      <td>10.214</td>\n",
       "      <td>1.4209</td>\n",
       "      <td>1.50266e+10</td>\n",
       "      <td>1.53707e+10</td>\n",
       "      <td>6.1825</td>\n",
       "      <td>7.0134</td>\n",
       "      <td>0.7811</td>\n",
       "      <td>1.3308e+09</td>\n",
       "      <td>1.14715e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.019045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000625.XSHE</th>\n",
       "      <td>9.0578</td>\n",
       "      <td>2.6843</td>\n",
       "      <td>5.04058e+10</td>\n",
       "      <td>6.14985e+10</td>\n",
       "      <td>4.6787</td>\n",
       "      <td>13.8136</td>\n",
       "      <td>0.49</td>\n",
       "      <td>2.8641e+10</td>\n",
       "      <td>2.97296e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.102683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000627.XSHE</th>\n",
       "      <td>-15.2546</td>\n",
       "      <td>2.3371</td>\n",
       "      <td>3.26215e+09</td>\n",
       "      <td>4.16056e+09</td>\n",
       "      <td>-1.6752</td>\n",
       "      <td>-2.352</td>\n",
       "      <td>-0.026</td>\n",
       "      <td>6.64998e+08</td>\n",
       "      <td>7.37243e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.141071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000630.XSHE</th>\n",
       "      <td>57.5058</td>\n",
       "      <td>1.1421</td>\n",
       "      <td>1.27234e+10</td>\n",
       "      <td>4.12988e+10</td>\n",
       "      <td>0.8359</td>\n",
       "      <td>3.5881</td>\n",
       "      <td>0.28</td>\n",
       "      <td>5.19117e+10</td>\n",
       "      <td>5.15217e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.006730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000651.XSHE</th>\n",
       "      <td>5.9396</td>\n",
       "      <td>2.4666</td>\n",
       "      <td>8.53031e+10</td>\n",
       "      <td>9.61232e+10</td>\n",
       "      <td>6.4434</td>\n",
       "      <td>26.114</td>\n",
       "      <td>2.52</td>\n",
       "      <td>8.87594e+10</td>\n",
       "      <td>8.10091e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.021505</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000671.XSHE</th>\n",
       "      <td>12.6725</td>\n",
       "      <td>2.8818</td>\n",
       "      <td>9.43805e+09</td>\n",
       "      <td>1.98038e+10</td>\n",
       "      <td>1.5208</td>\n",
       "      <td>15.4125</td>\n",
       "      <td>0.37</td>\n",
       "      <td>5.02815e+09</td>\n",
       "      <td>4.50771e+09</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.192474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000686.XSHE</th>\n",
       "      <td>19.2931</td>\n",
       "      <td>2.1206</td>\n",
       "      <td>1.56182e+10</td>\n",
       "      <td>2.69384e+10</td>\n",
       "      <td>2.4956</td>\n",
       "      <td>6.0839</td>\n",
       "      <td>0.45</td>\n",
       "      <td>1.39239e+09</td>\n",
       "      <td>8.56387e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>-0.068292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000709.XSHE</th>\n",
       "      <td>152.033</td>\n",
       "      <td>0.4391</td>\n",
       "      <td>1.86887e+10</td>\n",
       "      <td>1.04332e+11</td>\n",
       "      <td>0.0827</td>\n",
       "      <td>0.2746</td>\n",
       "      <td>0.011</td>\n",
       "      <td>8.27415e+10</td>\n",
       "      <td>8.27416e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000723.XSHE</th>\n",
       "      <td>14.9051</td>\n",
       "      <td>3.419</td>\n",
       "      <td>1.54955e+09</td>\n",
       "      <td>1.94043e+09</td>\n",
       "      <td>4.5559</td>\n",
       "      <td>6.8384</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.09477e+08</td>\n",
       "      <td>8.63775e+08</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.018018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000725.XSHE</th>\n",
       "      <td>6.4741</td>\n",
       "      <td>1.0194</td>\n",
       "      <td>2.88009e+10</td>\n",
       "      <td>6.34951e+10</td>\n",
       "      <td>2.2984</td>\n",
       "      <td>5.1155</td>\n",
       "      <td>0.101</td>\n",
       "      <td>2.47738e+10</td>\n",
       "      <td>2.31044e+10</td>\n",
       "      <td>2014-01-30</td>\n",
       "      <td>0.131448</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601618.XSHG</th>\n",
       "      <td>32.1927</td>\n",
       "      <td>2.7436</td>\n",
       "      <td>1.48867e+11</td>\n",
       "      <td>2.44498e+11</td>\n",
       "      <td>1.0941</td>\n",
       "      <td>6.2038</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1.454e+11</td>\n",
       "      <td>1.41277e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.227213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601628.XSHG</th>\n",
       "      <td>19.3179</td>\n",
       "      <td>2.5001</td>\n",
       "      <td>7.43644e+11</td>\n",
       "      <td>2.63029e+12</td>\n",
       "      <td>1.497</td>\n",
       "      <td>11.6097</td>\n",
       "      <td>1.2</td>\n",
       "      <td>4.24007e+11</td>\n",
       "      <td>3.79514e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.076018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601633.XSHG</th>\n",
       "      <td>12.863</td>\n",
       "      <td>3.3301</td>\n",
       "      <td>1.11444e+11</td>\n",
       "      <td>8.77384e+10</td>\n",
       "      <td>9.7974</td>\n",
       "      <td>18.5561</td>\n",
       "      <td>0.6802</td>\n",
       "      <td>5.26557e+10</td>\n",
       "      <td>4.54262e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.013926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601668.XSHG</th>\n",
       "      <td>7.5268</td>\n",
       "      <td>1.1563</td>\n",
       "      <td>1.86e+11</td>\n",
       "      <td>4.53794e+11</td>\n",
       "      <td>2.5515</td>\n",
       "      <td>12.7011</td>\n",
       "      <td>0.6</td>\n",
       "      <td>6.11342e+11</td>\n",
       "      <td>5.78214e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.022571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601669.XSHG</th>\n",
       "      <td>15.6166</td>\n",
       "      <td>1.8797</td>\n",
       "      <td>1.10037e+11</td>\n",
       "      <td>2.66782e+11</td>\n",
       "      <td>1.6088</td>\n",
       "      <td>9.3153</td>\n",
       "      <td>0.3605</td>\n",
       "      <td>1.44933e+11</td>\n",
       "      <td>1.38358e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.003749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601688.XSHG</th>\n",
       "      <td>12.7593</td>\n",
       "      <td>1.877</td>\n",
       "      <td>1.33729e+11</td>\n",
       "      <td>4.91888e+11</td>\n",
       "      <td>2.3281</td>\n",
       "      <td>14.5784</td>\n",
       "      <td>1.38</td>\n",
       "      <td>1.98823e+10</td>\n",
       "      <td>8.3822e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.056235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601718.XSHG</th>\n",
       "      <td>30.684</td>\n",
       "      <td>3.6936</td>\n",
       "      <td>4.72482e+10</td>\n",
       "      <td>5.23174e+10</td>\n",
       "      <td>4.5728</td>\n",
       "      <td>8.5317</td>\n",
       "      <td>0.27</td>\n",
       "      <td>1.50588e+10</td>\n",
       "      <td>1.5012e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.063667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601727.XSHG</th>\n",
       "      <td>54.183</td>\n",
       "      <td>3.9033</td>\n",
       "      <td>1.44013e+11</td>\n",
       "      <td>1.38318e+11</td>\n",
       "      <td>2.5159</td>\n",
       "      <td>5.3204</td>\n",
       "      <td>0.148</td>\n",
       "      <td>5.51619e+10</td>\n",
       "      <td>5.26694e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.027605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601766.XSHG</th>\n",
       "      <td>33.2578</td>\n",
       "      <td>3.7239</td>\n",
       "      <td>3.60212e+11</td>\n",
       "      <td>3.70772e+11</td>\n",
       "      <td>4.0822</td>\n",
       "      <td>8.5588</td>\n",
       "      <td>0.29</td>\n",
       "      <td>1.48555e+11</td>\n",
       "      <td>1.37816e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.017527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601788.XSHG</th>\n",
       "      <td>12.1787</td>\n",
       "      <td>2.3424</td>\n",
       "      <td>8.73147e+10</td>\n",
       "      <td>2.25141e+11</td>\n",
       "      <td>3.815</td>\n",
       "      <td>18.9144</td>\n",
       "      <td>1.7279</td>\n",
       "      <td>1.23502e+10</td>\n",
       "      <td>4.66079e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.026399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601800.XSHG</th>\n",
       "      <td>15.6509</td>\n",
       "      <td>1.6454</td>\n",
       "      <td>2.21917e+11</td>\n",
       "      <td>4.5847e+11</td>\n",
       "      <td>1.4883</td>\n",
       "      <td>7.9832</td>\n",
       "      <td>0.6</td>\n",
       "      <td>2.79833e+11</td>\n",
       "      <td>2.69324e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.022594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601818.XSHG</th>\n",
       "      <td>6.8664</td>\n",
       "      <td>0.9406</td>\n",
       "      <td>2.0212e+11</td>\n",
       "      <td>3.05329e+12</td>\n",
       "      <td>0.8213</td>\n",
       "      <td>12.0783</td>\n",
       "      <td>0.51</td>\n",
       "      <td>7.0047e+10</td>\n",
       "      <td>3.8436e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.020777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601857.XSHG</th>\n",
       "      <td>36.8436</td>\n",
       "      <td>1.3146</td>\n",
       "      <td>1.53738e+12</td>\n",
       "      <td>2.02998e+12</td>\n",
       "      <td>1.5023</td>\n",
       "      <td>2.6091</td>\n",
       "      <td>0.17</td>\n",
       "      <td>1.30510e+12</td>\n",
       "      <td>1.26196e+12</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.005963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601866.XSHG</th>\n",
       "      <td>-160.327</td>\n",
       "      <td>4.2678</td>\n",
       "      <td>1.01994e+11</td>\n",
       "      <td>1.05441e+11</td>\n",
       "      <td>-1.8457</td>\n",
       "      <td>-4.2467</td>\n",
       "      <td>-0.0219</td>\n",
       "      <td>2.74845e+10</td>\n",
       "      <td>2.82931e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.193585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601872.XSHG</th>\n",
       "      <td>25.0296</td>\n",
       "      <td>2.721</td>\n",
       "      <td>3.78911e+10</td>\n",
       "      <td>4.75463e+10</td>\n",
       "      <td>7.4517</td>\n",
       "      <td>13.5465</td>\n",
       "      <td>0.3354</td>\n",
       "      <td>4.55102e+09</td>\n",
       "      <td>3.11921e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.008389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601877.XSHG</th>\n",
       "      <td>20.0811</td>\n",
       "      <td>5.1143</td>\n",
       "      <td>3.58969e+10</td>\n",
       "      <td>4.21048e+10</td>\n",
       "      <td>10.7314</td>\n",
       "      <td>19.496</td>\n",
       "      <td>0.97</td>\n",
       "      <td>8.77497e+09</td>\n",
       "      <td>7.20729e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.055311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601888.XSHG</th>\n",
       "      <td>30.5684</td>\n",
       "      <td>4.4496</td>\n",
       "      <td>4.88021e+10</td>\n",
       "      <td>5.1184e+10</td>\n",
       "      <td>9.7084</td>\n",
       "      <td>12.5193</td>\n",
       "      <td>1.3507</td>\n",
       "      <td>1.57947e+10</td>\n",
       "      <td>1.39709e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.186438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601898.XSHG</th>\n",
       "      <td>-51.2885</td>\n",
       "      <td>0.9424</td>\n",
       "      <td>7.99497e+10</td>\n",
       "      <td>1.97925e+11</td>\n",
       "      <td>-0.5336</td>\n",
       "      <td>-1.9426</td>\n",
       "      <td>-0.13</td>\n",
       "      <td>4.48788e+10</td>\n",
       "      <td>4.70058e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.003320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601899.XSHG</th>\n",
       "      <td>36.0428</td>\n",
       "      <td>2.7004</td>\n",
       "      <td>7.45396e+10</td>\n",
       "      <td>9.71308e+10</td>\n",
       "      <td>2.1526</td>\n",
       "      <td>6.0797</td>\n",
       "      <td>0.078</td>\n",
       "      <td>6.11979e+10</td>\n",
       "      <td>5.96078e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.017347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601901.XSHG</th>\n",
       "      <td>18.0147</td>\n",
       "      <td>2.3155</td>\n",
       "      <td>7.80403e+10</td>\n",
       "      <td>1.94263e+11</td>\n",
       "      <td>3.0003</td>\n",
       "      <td>12.1443</td>\n",
       "      <td>0.48</td>\n",
       "      <td>7.99161e+09</td>\n",
       "      <td>2.75572e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.012660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601919.XSHG</th>\n",
       "      <td>98.5975</td>\n",
       "      <td>4.8553</td>\n",
       "      <td>1.18815e+11</td>\n",
       "      <td>1.84674e+11</td>\n",
       "      <td>0.7711</td>\n",
       "      <td>0.7706</td>\n",
       "      <td>0.03</td>\n",
       "      <td>4.96982e+10</td>\n",
       "      <td>5.27819e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.224420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601933.XSHG</th>\n",
       "      <td>49.2809</td>\n",
       "      <td>3.2259</td>\n",
       "      <td>3.94144e+10</td>\n",
       "      <td>4.13192e+10</td>\n",
       "      <td>3.2795</td>\n",
       "      <td>6.4014</td>\n",
       "      <td>0.08</td>\n",
       "      <td>3.1444e+10</td>\n",
       "      <td>3.07515e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.042317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601939.XSHG</th>\n",
       "      <td>6.2969</td>\n",
       "      <td>1.0678</td>\n",
       "      <td>1.44256e+12</td>\n",
       "      <td>1.8319e+13</td>\n",
       "      <td>1.0955</td>\n",
       "      <td>14.7075</td>\n",
       "      <td>0.77</td>\n",
       "      <td>4.57391e+11</td>\n",
       "      <td>2.14519e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.001724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601958.XSHG</th>\n",
       "      <td>1244.75</td>\n",
       "      <td>1.9989</td>\n",
       "      <td>2.57806e+10</td>\n",
       "      <td>2.33613e+10</td>\n",
       "      <td>0.0045</td>\n",
       "      <td>0.0237</td>\n",
       "      <td>0.001</td>\n",
       "      <td>7.58418e+09</td>\n",
       "      <td>7.63041e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>0.036295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601988.XSHG</th>\n",
       "      <td>7.1169</td>\n",
       "      <td>1.0329</td>\n",
       "      <td>1.20993e+12</td>\n",
       "      <td>1.64765e+13</td>\n",
       "      <td>0.8638</td>\n",
       "      <td>10.9634</td>\n",
       "      <td>0.43</td>\n",
       "      <td>3.56773e+11</td>\n",
       "      <td>1.78761e+11</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.024324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601989.XSHG</th>\n",
       "      <td>-406.752</td>\n",
       "      <td>3.1338</td>\n",
       "      <td>1.84718e+11</td>\n",
       "      <td>2.56939e+11</td>\n",
       "      <td>-0.343</td>\n",
       "      <td>-0.7424</td>\n",
       "      <td>-0.024</td>\n",
       "      <td>3.97721e+10</td>\n",
       "      <td>4.09465e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.065606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601991.XSHG</th>\n",
       "      <td>40.9071</td>\n",
       "      <td>1.5373</td>\n",
       "      <td>7.04101e+10</td>\n",
       "      <td>2.43024e+11</td>\n",
       "      <td>1.5346</td>\n",
       "      <td>8.0061</td>\n",
       "      <td>0.2695</td>\n",
       "      <td>4.6083e+10</td>\n",
       "      <td>4.16637e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.028350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601992.XSHG</th>\n",
       "      <td>26.2324</td>\n",
       "      <td>1.45</td>\n",
       "      <td>4.57412e+10</td>\n",
       "      <td>8.05162e+10</td>\n",
       "      <td>0.8684</td>\n",
       "      <td>3.3894</td>\n",
       "      <td>0.22</td>\n",
       "      <td>2.59027e+10</td>\n",
       "      <td>2.47853e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.019879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>601998.XSHG</th>\n",
       "      <td>8.217</td>\n",
       "      <td>1.1758</td>\n",
       "      <td>3.39676e+11</td>\n",
       "      <td>4.63789e+12</td>\n",
       "      <td>0.7606</td>\n",
       "      <td>11.8896</td>\n",
       "      <td>0.7</td>\n",
       "      <td>1.07453e+11</td>\n",
       "      <td>6.3722e+10</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.005505</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>603993.XSHG</th>\n",
       "      <td>85.9503</td>\n",
       "      <td>4.9651</td>\n",
       "      <td>8.59558e+10</td>\n",
       "      <td>6.06876e+10</td>\n",
       "      <td>2.0555</td>\n",
       "      <td>4.0126</td>\n",
       "      <td>0.041</td>\n",
       "      <td>3.17466e+09</td>\n",
       "      <td>2.73195e+09</td>\n",
       "      <td>2015-11-30</td>\n",
       "      <td>-0.123771</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>6136 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pe_ratio pb_ratio   market_cap           ev  \\\n",
       "000001.XSHE    7.126   0.9684  1.08537e+11  1.85809e+12   \n",
       "000002.XSHE   5.3769   1.0571  8.12905e+10  1.91646e+11   \n",
       "000008.XSHE  114.328   4.6166  2.60187e+09  2.91761e+09   \n",
       "000060.XSHE  28.8896   1.9053  1.13668e+10  1.91828e+10   \n",
       "000063.XSHE   8.2098   2.0244  4.56162e+10  9.63637e+10   \n",
       "000069.XSHE   7.1736   1.4932  3.56303e+10  6.55072e+10   \n",
       "000100.XSHE   6.2327   1.5054  2.13287e+10  5.31028e+10   \n",
       "000157.XSHE  111.858    0.935  3.89151e+10   7.0563e+10   \n",
       "000333.XSHE  10.7138   2.4057  7.90211e+10  9.53968e+10   \n",
       "000338.XSHE   8.4296    1.262  3.49879e+10  5.65886e+10   \n",
       "000402.XSHE   4.2384   0.6684  1.48932e+10  4.74655e+10   \n",
       "000413.XSHE  34.8362   3.5709    2.076e+10  1.65766e+10   \n",
       "000415.XSHE   7.9511   1.8681  1.05729e+10  3.54686e+10   \n",
       "000423.XSHE  17.3139   4.7263  2.36429e+10   2.6466e+10   \n",
       "000425.XSHE  24.4981   0.7322  1.41918e+10  3.43816e+10   \n",
       "000503.XSHE  11233.5  16.6486  2.09156e+10  1.83114e+10   \n",
       "000538.XSHE  20.8343   7.0612  6.37545e+10  8.18656e+10   \n",
       "000540.XSHE   3.9358   1.9948  6.78551e+09  2.09298e+10   \n",
       "000559.XSHE  16.8707   2.9299  1.01172e+10  1.26371e+10   \n",
       "000568.XSHE   9.2359   2.1721  2.29549e+10  2.99001e+10   \n",
       "000623.XSHE   10.214   1.4209  1.50266e+10  1.53707e+10   \n",
       "000625.XSHE   9.0578   2.6843  5.04058e+10  6.14985e+10   \n",
       "000627.XSHE -15.2546   2.3371  3.26215e+09  4.16056e+09   \n",
       "000630.XSHE  57.5058   1.1421  1.27234e+10  4.12988e+10   \n",
       "000651.XSHE   5.9396   2.4666  8.53031e+10  9.61232e+10   \n",
       "000671.XSHE  12.6725   2.8818  9.43805e+09  1.98038e+10   \n",
       "000686.XSHE  19.2931   2.1206  1.56182e+10  2.69384e+10   \n",
       "000709.XSHE  152.033   0.4391  1.86887e+10  1.04332e+11   \n",
       "000723.XSHE  14.9051    3.419  1.54955e+09  1.94043e+09   \n",
       "000725.XSHE   6.4741   1.0194  2.88009e+10  6.34951e+10   \n",
       "...              ...      ...          ...          ...   \n",
       "601618.XSHG  32.1927   2.7436  1.48867e+11  2.44498e+11   \n",
       "601628.XSHG  19.3179   2.5001  7.43644e+11  2.63029e+12   \n",
       "601633.XSHG   12.863   3.3301  1.11444e+11  8.77384e+10   \n",
       "601668.XSHG   7.5268   1.1563     1.86e+11  4.53794e+11   \n",
       "601669.XSHG  15.6166   1.8797  1.10037e+11  2.66782e+11   \n",
       "601688.XSHG  12.7593    1.877  1.33729e+11  4.91888e+11   \n",
       "601718.XSHG   30.684   3.6936  4.72482e+10  5.23174e+10   \n",
       "601727.XSHG   54.183   3.9033  1.44013e+11  1.38318e+11   \n",
       "601766.XSHG  33.2578   3.7239  3.60212e+11  3.70772e+11   \n",
       "601788.XSHG  12.1787   2.3424  8.73147e+10  2.25141e+11   \n",
       "601800.XSHG  15.6509   1.6454  2.21917e+11   4.5847e+11   \n",
       "601818.XSHG   6.8664   0.9406   2.0212e+11  3.05329e+12   \n",
       "601857.XSHG  36.8436   1.3146  1.53738e+12  2.02998e+12   \n",
       "601866.XSHG -160.327   4.2678  1.01994e+11  1.05441e+11   \n",
       "601872.XSHG  25.0296    2.721  3.78911e+10  4.75463e+10   \n",
       "601877.XSHG  20.0811   5.1143  3.58969e+10  4.21048e+10   \n",
       "601888.XSHG  30.5684   4.4496  4.88021e+10   5.1184e+10   \n",
       "601898.XSHG -51.2885   0.9424  7.99497e+10  1.97925e+11   \n",
       "601899.XSHG  36.0428   2.7004  7.45396e+10  9.71308e+10   \n",
       "601901.XSHG  18.0147   2.3155  7.80403e+10  1.94263e+11   \n",
       "601919.XSHG  98.5975   4.8553  1.18815e+11  1.84674e+11   \n",
       "601933.XSHG  49.2809   3.2259  3.94144e+10  4.13192e+10   \n",
       "601939.XSHG   6.2969   1.0678  1.44256e+12   1.8319e+13   \n",
       "601958.XSHG  1244.75   1.9989  2.57806e+10  2.33613e+10   \n",
       "601988.XSHG   7.1169   1.0329  1.20993e+12  1.64765e+13   \n",
       "601989.XSHG -406.752   3.1338  1.84718e+11  2.56939e+11   \n",
       "601991.XSHG  40.9071   1.5373  7.04101e+10  2.43024e+11   \n",
       "601992.XSHG  26.2324     1.45  4.57412e+10  8.05162e+10   \n",
       "601998.XSHG    8.217   1.1758  3.39676e+11  4.63789e+12   \n",
       "603993.XSHG  85.9503   4.9651  8.59558e+10  6.06876e+10   \n",
       "\n",
       "            return_on_asset_net_profit du_return_on_equity earnings_per_share  \\\n",
       "000001.XSHE                     0.6756             13.0164               1.43   \n",
       "000002.XSHE                     1.7804              9.3272               0.56   \n",
       "000008.XSHE                     2.7598              3.0992             0.0561   \n",
       "000060.XSHE                     0.9316               3.068               0.08   \n",
       "000063.XSHE                     0.5568               2.561               0.16   \n",
       "000069.XSHE                     4.0417             13.7533              0.399   \n",
       "000100.XSHE                     2.1248             10.2132              0.149   \n",
       "000157.XSHE                      4.387              9.2131               0.49   \n",
       "000333.XSHE                     7.2895             17.3695               3.73   \n",
       "000338.XSHE                     4.0573             10.4072               1.35   \n",
       "000402.XSHE                     2.1432              6.6307               0.44   \n",
       "000413.XSHE                     4.8108              7.0997               0.32   \n",
       "000415.XSHE                     1.7595              3.9744             0.4874   \n",
       "000423.XSHE                    15.7904             19.3468             1.3158   \n",
       "000425.XSHE                     2.3874              5.8692               0.51   \n",
       "000503.XSHE                     0.0588              0.1628             0.0023   \n",
       "000538.XSHE                    15.1017             22.5812               2.53   \n",
       "000540.XSHE                     2.5329             26.9999             0.5658   \n",
       "000559.XSHE                     3.5344              7.6187              0.184   \n",
       "000568.XSHE                    19.3343             27.4698               1.93   \n",
       "000623.XSHE                     6.1825              7.0134             0.7811   \n",
       "000625.XSHE                     4.6787             13.8136               0.49   \n",
       "000627.XSHE                    -1.6752              -2.352             -0.026   \n",
       "000630.XSHE                     0.8359              3.5881               0.28   \n",
       "000651.XSHE                     6.4434              26.114               2.52   \n",
       "000671.XSHE                     1.5208             15.4125               0.37   \n",
       "000686.XSHE                     2.4956              6.0839               0.45   \n",
       "000709.XSHE                     0.0827              0.2746              0.011   \n",
       "000723.XSHE                     4.5559              6.8384               0.11   \n",
       "000725.XSHE                     2.2984              5.1155              0.101   \n",
       "...                                ...                 ...                ...   \n",
       "601618.XSHG                     1.0941              6.2038               0.16   \n",
       "601628.XSHG                      1.497             11.6097                1.2   \n",
       "601633.XSHG                     9.7974             18.5561             0.6802   \n",
       "601668.XSHG                     2.5515             12.7011                0.6   \n",
       "601669.XSHG                     1.6088              9.3153             0.3605   \n",
       "601688.XSHG                     2.3281             14.5784               1.38   \n",
       "601718.XSHG                     4.5728              8.5317               0.27   \n",
       "601727.XSHG                     2.5159              5.3204              0.148   \n",
       "601766.XSHG                     4.0822              8.5588               0.29   \n",
       "601788.XSHG                      3.815             18.9144             1.7279   \n",
       "601800.XSHG                     1.4883              7.9832                0.6   \n",
       "601818.XSHG                     0.8213             12.0783               0.51   \n",
       "601857.XSHG                     1.5023              2.6091               0.17   \n",
       "601866.XSHG                    -1.8457             -4.2467            -0.0219   \n",
       "601872.XSHG                     7.4517             13.5465             0.3354   \n",
       "601877.XSHG                    10.7314              19.496               0.97   \n",
       "601888.XSHG                     9.7084             12.5193             1.3507   \n",
       "601898.XSHG                    -0.5336             -1.9426              -0.13   \n",
       "601899.XSHG                     2.1526              6.0797              0.078   \n",
       "601901.XSHG                     3.0003             12.1443               0.48   \n",
       "601919.XSHG                     0.7711              0.7706               0.03   \n",
       "601933.XSHG                     3.2795              6.4014               0.08   \n",
       "601939.XSHG                     1.0955             14.7075               0.77   \n",
       "601958.XSHG                     0.0045              0.0237              0.001   \n",
       "601988.XSHG                     0.8638             10.9634               0.43   \n",
       "601989.XSHG                     -0.343             -0.7424             -0.024   \n",
       "601991.XSHG                     1.5346              8.0061             0.2695   \n",
       "601992.XSHG                     0.8684              3.3894               0.22   \n",
       "601998.XSHG                     0.7606             11.8896                0.7   \n",
       "603993.XSHG                     2.0555              4.0126              0.041   \n",
       "\n",
       "                 revenue total_expense        date  next_month_return  \n",
       "000001.XSHE   3.7345e+10    2.2038e+10  2014-01-30          -0.023689  \n",
       "000002.XSHE  6.34153e+10     5.396e+10  2014-01-30          -0.089430  \n",
       "000008.XSHE  2.28189e+08   2.06793e+08  2014-01-30           0.121334  \n",
       "000060.XSHE  9.75875e+09   9.64269e+09  2014-01-30           0.079866  \n",
       "000063.XSHE  5.45575e+10   5.70062e+10  2014-01-30          -0.009043  \n",
       "000069.XSHE  1.72365e+10   1.32602e+10  2014-01-30          -0.077550  \n",
       "000100.XSHE  6.12241e+10   6.05763e+10  2014-01-30           0.112002  \n",
       "000157.XSHE  2.88971e+10   2.42468e+10  2014-01-30          -0.009907  \n",
       "000333.XSHE  9.39206e+10    8.7225e+10  2014-01-30          -0.169011  \n",
       "000338.XSHE  4.33327e+10   4.00661e+10  2014-01-30          -0.002849  \n",
       "000402.XSHE  1.01632e+10   8.23405e+09  2014-01-30          -0.065039  \n",
       "000413.XSHE   5.3758e+08   3.29781e+08  2014-01-30          -0.130489  \n",
       "000415.XSHE  4.49556e+09   3.62487e+09  2014-01-30          -0.005992  \n",
       "000423.XSHE  2.85154e+09   1.83781e+09  2014-01-30           0.012725  \n",
       "000425.XSHE  1.86193e+10   1.74667e+10  2014-01-30           0.010169  \n",
       "000503.XSHE  1.51521e+08   1.35794e+08  2014-01-30           0.048560  \n",
       "000538.XSHE  1.12287e+10   9.89682e+09  2014-01-30          -0.064576  \n",
       "000540.XSHE  5.37724e+09   4.45218e+09  2014-01-30          -0.064532  \n",
       "000559.XSHE  6.80359e+09   6.42877e+09  2014-01-30           0.885040  \n",
       "000568.XSHE  8.15632e+09   4.54271e+09  2014-01-30           0.026879  \n",
       "000623.XSHE   1.3308e+09   1.14715e+09  2014-01-30          -0.019045  \n",
       "000625.XSHE   2.8641e+10   2.97296e+10  2014-01-30          -0.102683  \n",
       "000627.XSHE  6.64998e+08   7.37243e+08  2014-01-30           0.141071  \n",
       "000630.XSHE  5.19117e+10   5.15217e+10  2014-01-30          -0.006730  \n",
       "000651.XSHE  8.87594e+10   8.10091e+10  2014-01-30          -0.021505  \n",
       "000671.XSHE  5.02815e+09   4.50771e+09  2014-01-30          -0.192474  \n",
       "000686.XSHE  1.39239e+09   8.56387e+08  2014-01-30          -0.068292  \n",
       "000709.XSHE  8.27415e+10   8.27416e+10  2014-01-30           0.000000  \n",
       "000723.XSHE  9.09477e+08   8.63775e+08  2014-01-30           0.018018  \n",
       "000725.XSHE  2.47738e+10   2.31044e+10  2014-01-30           0.131448  \n",
       "...                  ...           ...         ...                ...  \n",
       "601618.XSHG    1.454e+11   1.41277e+11  2015-11-30          -0.227213  \n",
       "601628.XSHG  4.24007e+11   3.79514e+11  2015-11-30           0.076018  \n",
       "601633.XSHG  5.26557e+10   4.54262e+10  2015-11-30          -0.013926  \n",
       "601668.XSHG  6.11342e+11   5.78214e+11  2015-11-30           0.022571  \n",
       "601669.XSHG  1.44933e+11   1.38358e+11  2015-11-30           0.003749  \n",
       "601688.XSHG  1.98823e+10    8.3822e+09  2015-11-30           0.056235  \n",
       "601718.XSHG  1.50588e+10    1.5012e+10  2015-11-30          -0.063667  \n",
       "601727.XSHG  5.51619e+10   5.26694e+10  2015-11-30           0.027605  \n",
       "601766.XSHG  1.48555e+11   1.37816e+11  2015-11-30          -0.017527  \n",
       "601788.XSHG  1.23502e+10   4.66079e+09  2015-11-30           0.026399  \n",
       "601800.XSHG  2.79833e+11   2.69324e+11  2015-11-30          -0.022594  \n",
       "601818.XSHG   7.0047e+10    3.8436e+10  2015-11-30          -0.020777  \n",
       "601857.XSHG  1.30510e+12   1.26196e+12  2015-11-30          -0.005963  \n",
       "601866.XSHG  2.74845e+10   2.82931e+10  2015-11-30          -0.193585  \n",
       "601872.XSHG  4.55102e+09   3.11921e+09  2015-11-30          -0.008389  \n",
       "601877.XSHG  8.77497e+09   7.20729e+09  2015-11-30          -0.055311  \n",
       "601888.XSHG  1.57947e+10   1.39709e+10  2015-11-30           0.186438  \n",
       "601898.XSHG  4.48788e+10   4.70058e+10  2015-11-30           0.003320  \n",
       "601899.XSHG  6.11979e+10   5.96078e+10  2015-11-30           0.017347  \n",
       "601901.XSHG  7.99161e+09   2.75572e+09  2015-11-30           0.012660  \n",
       "601919.XSHG  4.96982e+10   5.27819e+10  2015-11-30          -0.224420  \n",
       "601933.XSHG   3.1444e+10   3.07515e+10  2015-11-30           0.042317  \n",
       "601939.XSHG  4.57391e+11   2.14519e+11  2015-11-30           0.001724  \n",
       "601958.XSHG  7.58418e+09   7.63041e+09  2015-11-30           0.036295  \n",
       "601988.XSHG  3.56773e+11   1.78761e+11  2015-11-30          -0.024324  \n",
       "601989.XSHG  3.97721e+10   4.09465e+10  2015-11-30          -0.065606  \n",
       "601991.XSHG   4.6083e+10   4.16637e+10  2015-11-30          -0.028350  \n",
       "601992.XSHG  2.59027e+10   2.47853e+10  2015-11-30          -0.019879  \n",
       "601998.XSHG  1.07453e+11    6.3722e+10  2015-11-30          -0.005505  \n",
       "603993.XSHG  3.17466e+09   2.73195e+09  2015-11-30          -0.123771  \n",
       "\n",
       "[6136 rows x 11 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 五、特征值和目标值处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def mad(factor):\n",
    "  \"\"\"3倍中位数去极值\n",
    "  \"\"\"\n",
    "  # 求出因子值的中位数\n",
    "  med = np.median(factor)\n",
    "\n",
    "  # 求出因子值与中位数的差值，进行绝对值\n",
    "  mad = np.median(np.abs(factor - med))\n",
    "\n",
    "  # 定义几倍的中位数上下限\n",
    "  high = med + (3 * 1.4826 * mad)\n",
    "  low = med - (3 * 1.4826 * mad)\n",
    "\n",
    "  # 替换上下限以外的值\n",
    "  factor = np.where(factor > high, high, factor)\n",
    "  factor = np.where(factor < low, low, factor)\n",
    "  return factor\n",
    "\n",
    "def stand(factor):\n",
    "    \"\"\"标准化\n",
    "    \"\"\"\n",
    "    mean = np.mean(factor)\n",
    "    std = np.std(factor)\n",
    "    return (factor - mean)/std"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = all_data['next_month_return']\n",
    "x = all_data.drop(['next_month_return', 'date'], axis=1)\n",
    "x_market_cap = x['market_cap']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pe_ratio</th>\n",
       "      <th>pb_ratio</th>\n",
       "      <th>market_cap</th>\n",
       "      <th>ev</th>\n",
       "      <th>return_on_asset_net_profit</th>\n",
       "      <th>du_return_on_equity</th>\n",
       "      <th>earnings_per_share</th>\n",
       "      <th>revenue</th>\n",
       "      <th>total_expense</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>000001.XSHE</th>\n",
       "      <td>7.126</td>\n",
       "      <td>0.9684</td>\n",
       "      <td>1.08537e+11</td>\n",
       "      <td>1.85809e+12</td>\n",
       "      <td>0.6756</td>\n",
       "      <td>13.0164</td>\n",
       "      <td>1.43</td>\n",
       "      <td>3.7345e+10</td>\n",
       "      <td>2.2038e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000002.XSHE</th>\n",
       "      <td>5.3769</td>\n",
       "      <td>1.0571</td>\n",
       "      <td>8.12905e+10</td>\n",
       "      <td>1.91646e+11</td>\n",
       "      <td>1.7804</td>\n",
       "      <td>9.3272</td>\n",
       "      <td>0.56</td>\n",
       "      <td>6.34153e+10</td>\n",
       "      <td>5.396e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000008.XSHE</th>\n",
       "      <td>114.328</td>\n",
       "      <td>4.6166</td>\n",
       "      <td>2.60187e+09</td>\n",
       "      <td>2.91761e+09</td>\n",
       "      <td>2.7598</td>\n",
       "      <td>3.0992</td>\n",
       "      <td>0.0561</td>\n",
       "      <td>2.28189e+08</td>\n",
       "      <td>2.06793e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000060.XSHE</th>\n",
       "      <td>28.8896</td>\n",
       "      <td>1.9053</td>\n",
       "      <td>1.13668e+10</td>\n",
       "      <td>1.91828e+10</td>\n",
       "      <td>0.9316</td>\n",
       "      <td>3.068</td>\n",
       "      <td>0.08</td>\n",
       "      <td>9.75875e+09</td>\n",
       "      <td>9.64269e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000063.XSHE</th>\n",
       "      <td>8.2098</td>\n",
       "      <td>2.0244</td>\n",
       "      <td>4.56162e+10</td>\n",
       "      <td>9.63637e+10</td>\n",
       "      <td>0.5568</td>\n",
       "      <td>2.561</td>\n",
       "      <td>0.16</td>\n",
       "      <td>5.45575e+10</td>\n",
       "      <td>5.70062e+10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            pe_ratio pb_ratio   market_cap           ev  \\\n",
       "000001.XSHE    7.126   0.9684  1.08537e+11  1.85809e+12   \n",
       "000002.XSHE   5.3769   1.0571  8.12905e+10  1.91646e+11   \n",
       "000008.XSHE  114.328   4.6166  2.60187e+09  2.91761e+09   \n",
       "000060.XSHE  28.8896   1.9053  1.13668e+10  1.91828e+10   \n",
       "000063.XSHE   8.2098   2.0244  4.56162e+10  9.63637e+10   \n",
       "\n",
       "            return_on_asset_net_profit du_return_on_equity earnings_per_share  \\\n",
       "000001.XSHE                     0.6756             13.0164               1.43   \n",
       "000002.XSHE                     1.7804              9.3272               0.56   \n",
       "000008.XSHE                     2.7598              3.0992             0.0561   \n",
       "000060.XSHE                     0.9316               3.068               0.08   \n",
       "000063.XSHE                     0.5568               2.561               0.16   \n",
       "\n",
       "                 revenue total_expense  \n",
       "000001.XSHE   3.7345e+10    2.2038e+10  \n",
       "000002.XSHE  6.34153e+10     5.396e+10  \n",
       "000008.XSHE  2.28189e+08   2.06793e+08  \n",
       "000060.XSHE  9.75875e+09   9.64269e+09  \n",
       "000063.XSHE  5.45575e+10   5.70062e+10  "
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 1、特征值处理\n",
    "# 去极值、标准化、中性化\n",
    "for name in x.columns:\n",
    "  x[name] = mad(x[name])\n",
    "  x[name] = stand(x[name])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pe_ratio</th>\n",
       "      <th>pb_ratio</th>\n",
       "      <th>market_cap</th>\n",
       "      <th>ev</th>\n",
       "      <th>return_on_asset_net_profit</th>\n",
       "      <th>du_return_on_equity</th>\n",
       "      <th>earnings_per_share</th>\n",
       "      <th>revenue</th>\n",
       "      <th>total_expense</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>000001.XSHE</th>\n",
       "      <td>-0.770255</td>\n",
       "      <td>-1.04225</td>\n",
       "      <td>1.5755</td>\n",
       "      <td>1.77523</td>\n",
       "      <td>-0.7284</td>\n",
       "      <td>1.01763</td>\n",
       "      <td>2.18913</td>\n",
       "      <td>1.61114</td>\n",
       "      <td>1.22293</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000002.XSHE</th>\n",
       "      <td>-0.824801</td>\n",
       "      <td>-1.00937</td>\n",
       "      <td>0.855849</td>\n",
       "      <td>1.77523</td>\n",
       "      <td>-0.319151</td>\n",
       "      <td>0.427776</td>\n",
       "      <td>0.670299</td>\n",
       "      <td>1.61114</td>\n",
       "      <td>1.55947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000008.XSHE</th>\n",
       "      <td>1.89736</td>\n",
       "      <td>0.310218</td>\n",
       "      <td>-1.22255</td>\n",
       "      <td>-1.13321</td>\n",
       "      <td>0.0436467</td>\n",
       "      <td>-0.56799</td>\n",
       "      <td>-0.850114</td>\n",
       "      <td>-0.96684</td>\n",
       "      <td>-0.975081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000060.XSHE</th>\n",
       "      <td>-0.0915574</td>\n",
       "      <td>-0.694924</td>\n",
       "      <td>-0.991041</td>\n",
       "      <td>-0.854831</td>\n",
       "      <td>-0.63357</td>\n",
       "      <td>-0.572979</td>\n",
       "      <td>-0.778001</td>\n",
       "      <td>-0.200865</td>\n",
       "      <td>-0.0250552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000063.XSHE</th>\n",
       "      <td>-0.736457</td>\n",
       "      <td>-0.650771</td>\n",
       "      <td>-0.0864143</td>\n",
       "      <td>0.466137</td>\n",
       "      <td>-0.772406</td>\n",
       "      <td>-0.654041</td>\n",
       "      <td>-0.536617</td>\n",
       "      <td>1.61114</td>\n",
       "      <td>1.55947</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              pe_ratio  pb_ratio market_cap        ev  \\\n",
       "000001.XSHE  -0.770255  -1.04225     1.5755   1.77523   \n",
       "000002.XSHE  -0.824801  -1.00937   0.855849   1.77523   \n",
       "000008.XSHE    1.89736  0.310218   -1.22255  -1.13321   \n",
       "000060.XSHE -0.0915574 -0.694924  -0.991041 -0.854831   \n",
       "000063.XSHE  -0.736457 -0.650771 -0.0864143  0.466137   \n",
       "\n",
       "            return_on_asset_net_profit du_return_on_equity earnings_per_share  \\\n",
       "000001.XSHE                    -0.7284             1.01763            2.18913   \n",
       "000002.XSHE                  -0.319151            0.427776           0.670299   \n",
       "000008.XSHE                  0.0436467            -0.56799          -0.850114   \n",
       "000060.XSHE                   -0.63357           -0.572979          -0.778001   \n",
       "000063.XSHE                  -0.772406           -0.654041          -0.536617   \n",
       "\n",
       "              revenue total_expense  \n",
       "000001.XSHE   1.61114       1.22293  \n",
       "000002.XSHE   1.61114       1.55947  \n",
       "000008.XSHE  -0.96684     -0.975081  \n",
       "000060.XSHE -0.200865    -0.0250552  \n",
       "000063.XSHE   1.61114       1.55947  "
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for name in x.columns:\n",
    "  \n",
    "    if name == \"market_cap\":\n",
    "      continue\n",
    "\n",
    "    # 准备特征值、目标值\n",
    "    # x_market_cap\n",
    "    y_factor = x[name]\n",
    "\n",
    "    # 线性回归方程建立\n",
    "    lr = LinearRegression()\n",
    "\n",
    "    lr.fit(x_market_cap.values.reshape(-1, 1), y_factor)\n",
    "\n",
    "    y_predict = lr.predict(x_market_cap.values.reshape(-1, 1))\n",
    "\n",
    "    # 得出真实值与预测之间的误差当做新的因子值\n",
    "    x[name] = y_factor - y_predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 收益率目标值y \n",
    "y = stand(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 六、建立特征值因子数据（处理过的） 与目标值（标准化） 下期收益率 之间的回归方程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lr = LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.fit(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.02953221, -0.04920124, -0.10791485,  0.00801783, -0.03613599,\n",
       "        0.1310877 , -0.03030564,  0.40286239, -0.30166898])"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
